Real-time video has become increasingly popular to users. However, when there is limited bandwidth available for the real-time video, some type of compression needs to be performed in order to reduce the amount of data that is transmitted on the limited bandwidth. One type of compression involves temporal sub-sampling. Temporal sub-sampling reduces the amount of data by skipping specific frames, such as skipping one frame in every two frames. When a decoder receives a bitstream encoded using temporal sub-sampling, the decoder is responsible for recovering (i.e., interpolating) the skipped frames in order to restore the original frame rate.
There are various approaches for interpolating the skipped frames. One approach, commonly called frame repetition, repeats each frame that is sent. The repeated frame then fills in the time slot for the skipped frame. However, frame repetition introduces jerky artifacts quite noticeable to the users. Another approach, commonly called frame averaging, averages a pixel from a previous frame and a current frame to obtain a corresponding pixel for the interpolated frame. However, frame averaging introduces blurry artifacts because it does not consider motion.
Recently, another approach has gained increasing popularity. This approach, commonly called motion compensated frame interpolation (MCFI), considers motion during its interpolation. Motion compensated frame interpolation performs motion estimation on the previous frame and the current frame to obtain a motion vector. The motion compensated frame interpolation shifts the corresponding pixels of the previous frame and the current frame by half of the obtained motion vector along opposite directions and then interpolates the skipped frame by averaging the shifted pixels in the previous frame and the current frame. Because motion compensated frame interpolation uses motion vectors to recover skipped frames, there has been considerable research efforts directed at improving the accuracy of estimating motion vectors.
Even with these advances, there is still a need for improved interpolation techniques that result in higher quality frame interpolation with minimal computing complexity.